OU-1472: run only changed tests by PeterYurkovich · Pull Request #1134 · openshift/monitoring-plugin

PeterYurkovich · 2026-08-12T19:39:27Z

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

Summary by CodeRabbit

New Features
- Added selective Cypress test execution based on changed files, running only relevant tagged tests when possible.
- Added fallback coverage for global or unmapped changes and skipped runs when no applicable source files changed.
Documentation
- Updated Cypress testing guidance and quick commands to use the new monitoring test workflow.
- Added instructions for direct spec execution and updated debugging commands.
Bug Fixes
- Corrected a Cypress support import path.

openshift-merge-bot · 2026-08-12T19:39:30Z

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

openshift-ci-robot · 2026-08-12T19:39:31Z

@PeterYurkovich: This pull request references OU-1472 which is a valid jira issue.

Details

In response to this:

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-08-12T19:41:04Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: PeterYurkovich

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [PeterYurkovich]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jgbernalp · 2026-08-17T10:43:18Z

@@ -0,0 +1,62 @@
+#!/bin/bash
+
+CHANGED_FILES=$(git diff --name-only origin/main...HEAD)


The target branch might be different we should provide it from env defaulting to main

we should add error handling, in shallow checkouts the main branch might not exist.

jgbernalp · 2026-08-17T10:50:03Z

-    "test-cypress-perses-dev": "node --max-old-space-size=4096 ./node_modules/.bin/cypress run --browser chrome --headless --env grepTags='@perses-dashboards --@xfail'",
-    "test-cypress-perses": "node --max-old-space-size=4096 ./node_modules/.bin/cypress run --browser chrome --headless --env grepTags='@perses-dashboards --@smoke --@flaky --@xfail'",
-    "test-cypress-perses-ivt": "node --max-old-space-size=4096 ./node_modules/.bin/cypress run --browser chrome --headless --env grepTags='@perses-dashboards --@smoke --@flaky --@xfail'",
+    "test-cypress-monitoring": "../scripts/run-selective-e2e.sh",


Is this script in a different PR or should be ./cypress/run-changed.sh

jgbernalp · 2026-08-17T10:52:37Z

+if [ "$RUN_ALL" = true ]; then
+    echo "Changes affect global scope or unmapped features. Running ALL monitoring tests..."
+
+    DEFAULT_TAGS="@alerting @legacy-dashboards @metrics @targets acm-alerting @cluster-health-analyzer @perses-dashboards --@flaky --@xfail"


Suggested change

DEFAULT_TAGS="@alerting @legacy-dashboards @metrics @targets acm-alerting @cluster-health-analyzer @perses-dashboards --@flaky --@xfail"

DEFAULT_TAGS="@alerting @legacy-dashboards @metrics @targets @acm-alerting @cluster-health-analyzer @perses-dashboards --@flaky --@xfail"

jgbernalp · 2026-08-17T10:53:43Z

    "test": "npm run cypress:run:ci",
-    "test-cypress-console": "./node_modules/.bin/cypress open --browser chrome",
-    "test-cypress-console-headless": "node --max-old-space-size=4096 ./node_modules/.bin/cypress run --browser chrome --headless",
-    "test-cypress-monitoring": "node --max-old-space-size=4096 ./node_modules/.bin/cypress run --browser chrome --headless --env grepTags='@alerting+@legacy-dashboards+@metrics+@targets --@flaky --@xfail'",


Do we need to update the cypress docs with these removals?

jgbernalp · 2026-08-17T10:57:47Z

+CHANGED_FILES=$(git diff --name-only origin/main...HEAD)
+
+RUN_ALL=false
+declare -A TAGS_TO_RUN


We might need to change this into something POSIX compatible, I got the following issue while trying to run on mac:

./cypress/run-changed.sh: line 6: declare: -A: invalid option declare: usage: declare [-afFirtx] [-p] [name[=value] ...]

jgbernalp · 2026-08-17T10:59:27Z

+done
+
+if [ "$RUN_ALL" = true ]; then
+    echo "Changes affect global scope or unmapped features. Running ALL monitoring tests..."


unmapped features already exits

Suggested change

echo "Changes affect global scope or unmapped features. Running ALL monitoring tests..."

echo "Changes affect global scope. Running ALL monitoring tests..."

jgbernalp · 2026-08-17T11:00:59Z

+FEATURE_MAP["perses-dashboards"]="@perses-dashboards"
+FEATURE_MAP["targets"]="@targets"
+
+for FILE in $CHANGED_FILES; do


not sure if we have file paths with white spaces, but this will split on white space

coderabbitai · 2026-09-09T16:49:42Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 696cf3a1-d04a-45fd-9948-a8dd27d2f07c

📥 Commits

Reviewing files that changed from the base of the PR and between 7d83ffb and ceff405.

📒 Files selected for processing (5)

AGENTS.md
web/cypress/CYPRESS_TESTING_GUIDE.md
web/cypress/run-changed.sh
web/cypress/support/alerts/alerts_regressions.cy.ts
web/package.json

Included review availability: Your plan provides up to 12 included reviews per hour; 8 remain after this review.

Walkthrough

The pull request adds changed-file-based Cypress monitoring selection, preserves the base Cypress command, updates command documentation, replaces obsolete quick commands, and fixes one Cypress support import path.

Changes

Cypress monitoring execution

Layer / File(s)	Summary
Selective monitoring runner `web/cypress/run-changed.sh`, `web/package.json`	The monitoring command maps changed feature directories to Cypress tags and runs the selected tests through a preserved base command.
Command guidance and Cypress compatibility `AGENTS.md`, `web/cypress/CYPRESS_TESTING_GUIDE.md`, `web/cypress/support/alerts/alerts_regressions.cy.ts`	Quick commands and debugging instructions use the web-rooted monitoring command. COO execution uses tagged Cypress commands. The alert regression imports `listPage` through a relative path.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 56384

This change introduces selective Cypress monitoring execution while retaining the base test command. No current merge-blocking risk is identified.

Suggested reviewers: jgbernalp

Sequence Diagram(s)

sequenceDiagram
  participant Git
  participant run-changed.sh
  participant npm
  participant Cypress
  Git->>run-changed.sh: Detect changed files
  run-changed.sh->>run-changed.sh: Map changes to Cypress tags
  run-changed.sh->>npm: Invoke monitoring test command
  npm->>Cypress: Run selected tagged tests

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 2 files. (3 skipped: 3 …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PASS: The pull request does not add or modify Ginkgo tests. The verified diff changes only Cypress documentation, shell/package scripts, and a Cypress import. Repository searches found no Ginkgo decla…
Test Structure And Quality	✅ Passed	PASS: The custom check applies to Ginkgo test code, but this PR changes no Go or Ginkgo files. The five changed paths are documentation, shell/package configuration, and one Cypress import-only change…
Microshift Test Compatibility	✅ Passed	The MicroShift compatibility check is not applicable. The pull request changes only Cypress scripts, Cypress TypeScript support code, package metadata, and documentation. The verified pull-request dif…
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The check is not applicable. The PR diff from `origin/main` contains no Go files and no new Ginkgo e2e tests. The changed test files are Cypress TypeScript files (`*.cy.ts`), plus scripts, documentati…
Topology-Aware Scheduling Compatibility	✅ Passed	PASS: The PR changes only Cypress scripts, Cypress test import code, documentation, and `web/package.json`. The verified PR diff contains no deployment manifests, operator code, controllers, workload …
Ote Binary Stdout Contract	✅ Passed	PASS: The pull request changes only Cypress documentation, a shell test-selection script, package metadata, and a TypeScript import. The diff contains no Go or OTE binary code and no main/init/TestMai…
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PASS: The pull request adds or changes Cypress scripts, documentation, package scripts, and one Cypress import. The commit range contains no added Go files or Ginkgo declarations such as `It()`, `Desc…
No-Weak-Crypto	✅ Passed	PASS. The PR changes only Cypress selection scripts, package commands, documentation, and an import path. The focused diff (`c54fb25`^..HEAD) adds no MD5, SHA1, DES, 3DES, RC4, Blowfish, or ECB usage. A…
Container-Privileges	✅ Passed	No custom-check failure was introduced. The PR diff adds Cypress scripts, documentation, test code, and fixture moves, but no added `privileged: true`, `hostPID`, `hostNetwork`, `hostIPC`, `SYS_ADMIN`…
No-Sensitive-Data-In-Logs	✅ Passed	No sensitive-data logging was introduced. The added shell script logs only changed repository paths, feature-directory errors, fixed status text, and Cypress tag names. The other code change only upda…
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: running only tests affected by changed files.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 2 files. (3 skipped: 3 unsupported.)

Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch run-changes

_{Comment @coderabbitai help to get the list of available commands.}

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 12, 2026

openshift-ci Bot requested review from etmurasaki and jgbernalp August 12, 2026 19:40

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 12, 2026

PeterYurkovich force-pushed the run-changes branch from 4e1102d to 3335c07 Compare August 12, 2026 20:01

PeterYurkovich force-pushed the run-changes branch from 3335c07 to 217d32c Compare August 12, 2026 20:06

jgbernalp reviewed Aug 17, 2026

View reviewed changes

PeterYurkovich force-pushed the run-changes branch 2 times, most recently from 2c3cc81 to 88ad602 Compare August 17, 2026 13:35

PeterYurkovich force-pushed the run-changes branch from 88ad602 to 49d7ac8 Compare August 17, 2026 19:07

PeterYurkovich force-pushed the run-changes branch 3 times, most recently from 489aca6 to 3ce7387 Compare August 19, 2026 21:21

PeterYurkovich force-pushed the run-changes branch 2 times, most recently from 3b1dbb7 to 6e7f913 Compare August 31, 2026 21:56

PeterYurkovich force-pushed the run-changes branch 2 times, most recently from 8c07d23 to b9236d2 Compare September 1, 2026 16:10

PeterYurkovich force-pushed the run-changes branch from b9236d2 to f3bd9eb Compare September 1, 2026 16:15

PeterYurkovich force-pushed the run-changes branch from f3bd9eb to dd928b5 Compare September 1, 2026 16:26

PeterYurkovich force-pushed the run-changes branch from 42970ca to 8ef35ed Compare September 1, 2026 16:42

PeterYurkovich force-pushed the run-changes branch 2 times, most recently from 300c476 to f0cac24 Compare September 1, 2026 19:21

PeterYurkovich force-pushed the run-changes branch from f0cac24 to 5fbeb01 Compare September 1, 2026 21:05

PeterYurkovich force-pushed the run-changes branch from 5fbeb01 to 9bbf6e1 Compare September 1, 2026 23:05

PeterYurkovich force-pushed the run-changes branch from 9bbf6e1 to a44cc93 Compare September 1, 2026 23:07

PeterYurkovich added 3 commits September 9, 2026 12:56

feat: run only changed tests

0b43e70

fix: remove last instance of 'cypress/`

479cb63

chore: update documentation

56384dc

Navigation Menu

Uh oh!

OU-1472: run only changed tests - #1134

OU-1472: run only changed tests#1134
PeterYurkovich wants to merge 3 commits into
move-viewsfrom
run-changes

PeterYurkovich commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

openshift-merge-bot Bot commented Aug 12, 2026

Uh oh!

openshift-ci-robot commented Aug 12, 2026 •

edited by openshift-ci Bot

Loading

Uh oh!

openshift-ci Bot commented Aug 12, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

jgbernalp Aug 17, 2026

Uh oh!

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,62 @@
		#!/bin/bash

		CHANGED_FILES=$(git diff --name-only origin/main...HEAD)

	DEFAULT_TAGS="@alerting @legacy-dashboards @metrics @targets acm-alerting @cluster-health-analyzer @perses-dashboards --@flaky --@xfail"
	DEFAULT_TAGS="@alerting @legacy-dashboards @metrics @targets @acm-alerting @cluster-health-analyzer @perses-dashboards --@flaky --@xfail"

	echo "Changes affect global scope or unmapped features. Running ALL monitoring tests..."
	echo "Changes affect global scope. Running ALL monitoring tests..."

Uh oh!

Conversation

PeterYurkovich commented Aug 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

openshift-merge-bot Bot commented Aug 12, 2026

Uh oh!

openshift-ci-robot commented Aug 12, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci Bot commented Aug 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

PeterYurkovich commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

openshift-ci-robot commented Aug 12, 2026 •

edited by openshift-ci Bot

Loading

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading